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Audience 


REALSYSTEM SERVER BEST PRACTICES 


This white paper provides the “best practices”—or the most useful techniques 
and approaches—for deploying RealNetworks® RealSystem Server® across a 
network. It is targeted at network managers, system architects, and 
administrators who are interested in using RealServer for the following 
purposes: 


- Building a streaming media network infrastructure. 


- Delivering streamed content rapidly and efficiently, regardless of the data 
transmission rates, or bandwidths, of the connections. 


This white paper assumes a solid understanding of RealServer, network 
architecture, and the leading operating systems, as well as familiarity with the 
process of creating streaming media files. For a more thorough understanding 
of these technologies, see the links in the Additional Resources section later 
in this paper. 

Because of the bandwidth-hungry nature of streaming media, the consistent 
and reliable delivery of streamed files—often over complex network paths—is 
imperative. Understanding how to calculate your hardware and software 
system requirements is crucial to building an effective, scalable network that 
can respond gracefully to sudden bursts of traffic. While RealNetworks 
provides reasonable numbers for basic planning on the three most popular 
platforms, in the Memory Usage section, there are numerous variables that 
can affect your real-world deployment and performance. 
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Performance Tuning 


System-Specific Tips 
Even though RealSystem Server is available for a wide range of platforms, the 


tuning tips presented in this section focus on the three that are most widely 
used: 


- Linux RedHat version 6.2 
- Windows NT version 4.0 


- Solaris versions 2.6 and 2.7 


Linux Redhat 6.2-Specific Tuning Tips 


Be sure to download and apply the latest patches, which are available at 
http://www.redhat.com/support/docs/rhl62.html. For other UNIX 
kernels (such as Debian, for example), refer to your system documentation. 


We recommend that you upgrade to the latest stable Linux 2.2.x kernel, which 
is available at http://www.kernel.org. By using the latest version of this 
kernel with the glibc2 library, you will experience improved performance in 
the following areas: 


- The Linux 2.2.x kernel now has improved symmetric multiprocessing 
(SMP) capability, which is the technology that enables it to use multiple 
processors. 


- The glibc2 library makes extensive use of multiple threads, which work 
very well with RealServer’s multithreaded model. 


We do not recommend using any Linux 2.3 or 2.4 kernels, as RealServer has 
not been thoroughly tested with these newer kernels. 


After you have upgraded the Linux 2.2.x kernel, use the following instructions 
to make specific file descriptor changes for RealServer 8. 


Editing Linux 2.2.x Kernel File Descriptors 
Append the following lines of code to /etc/rc.d/rc.local: 


echo "32768" >/proc/sys/fs/inode-max 

echo "19000" >/proc/ sys/ fs/file-max 

echo "1" >/proc/sys/net/ipv4/tcp_syncookies 

echo "2" >/proc/sys/net/ipv4/tcp_retries1 

echo "8192" >/proc/sys/net/ipv4/tcp_max_syn_backlog 
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Note that there is no need to set the streamersessioncapacity variable. 


Shell Limit 
Log on as root, and then set ‘ulimit -n 32768' in the shell that starts RealServer. 


By default, RealServer reserves 256 MB of memory (RAM) on your computer. 


Windows NT-Specific Tuning Tips 


Be sure to download and apply the latest patches, which are available at 
http://www.microsoft.com/ntserver/. 


Solaris 2.6- and 2.7-Specific Tuning Tips 


Be sure to download and apply the latest patches, which are available at 
http://sunsolve.sun.com/. 


On Solaris 2.6 and 2.7 systems, RealServer is designed to “prereserve” a large 
amount of memory and then use memory from that block. If you are running 
only RealServer on your computer, you can increase the amount of available 
memory by using the -m command, which you can set on RealServer’s startup 
screen. Do not reserve more than 75 percent of your computer’s total memory 
for RealServer, or its performance will suffer. 


Editing Solaris Kernel File Descriptors 

On Solaris 2.6, the number of file descriptors for the select function call is 
hard-coded at 1024 and cannot be changed. To get around this, the RealServer 
core will automatically generate another streamer process that has 800 file 
descriptors. 


Shell Limit 
On Solaris 2.7, log on as root, and then set ‘ulimit -n 32768' in the shell that 
starts RealServer. 


Running RealServer Alone 


Regardless of the platform being used, RealServer performs best when it runs 
on a dedicated computer—that is, one that has nothing else installed on it. 
Therefore, avoid installing Web servers or other applications on the same 
computer as RealServer. 
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The following table lists the network interface cards (NICs) that have been 
identified as working well with RealNetworks RealSystem®. 


Recommended Network Interface Cards 


Platform Card Product Name 


Intel EtherExpress Pro 





Windows NT or 2000 Intel EtherExpress Pro 


Solaris Sun built-in 





Memory Usage 


RealServer uses a multithreaded, CPU-sharing model. Threads share their 
environment and metadata information with each other, so statefulness does 
not need to be tracked as threads change. Multithreading is particularly well- 
suited to client/server architecture. When a client tries connecting to the 
server, the system manager creates a new thread to manage that client, while 
another thread listens for additional client connections. 


In working with streaming media, a general rule of thumb is to provide 12 KB 
of random-access memory (RAM) for every kilobit per second (Kbps) of 
streaming. RealServer performs best when it is running on a system with more 
than one processor, or CPU. This is because within a multiprocessor system, 
the scheduler can carry out the same process over multiple processors by 
simultaneously running a different thread on each one. 


Running RealServer on a multiprocessor system is also a wise choice if you 
decide to use the Ad Serving extension to RealServer, or if you are generating 
numerous SMIL media files (SMIL stands for Synchronized Multimedia 
Integration Language). 


Adding more memory to a computer will, of course, enhance its performance. 
With streaming media especially, to accommodate peak surges in network 
traffic, install more memory than you think you would ever need. If there is 
not enough RAM available for all of the information the CPU needs, the 
computer has to create a virtual memory file, reserving space on the hard disk 
to simulate additional RAM. This process, known as “swapping,” slows the 
system down. 


Memory and processors affect RealServer performance in somewhat different 
ways. Adding memory enables RealServer to handle more information at any 
given time, whereas adding processors enables RealServer to process 
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information faster. Thus, if you add processing power to a system, additional 
memory will help the processors perform at their optimal level. 


The following table provides real-world memory-usage benchmarks for 
RealServer. It uses the metric of estimated bandwidth being served to calculate 
computer memory recommendations across the most-used platforms. Note 
that although these configurations are intended to reflect real-world usage, 
your results may vary. 


Recommended RealServer Computer Configurations 
































Estimated Estimated 
Streams bandwidth 
Served served Linux Windows NT 4.0 Solaris 2.6 Solaris 2.7 
45 10 MB Pentium II 350, | Pentium III 500, | Ultra 10, Ultra 10, 
512 MB RAM 512 MB RAM 512 MB RAM 512 MB RAM 
134 30 MB Pentium III 750, | Pentium III 750, | Ultra 30, Ultra 30, 
768 MB RAM 768 MB 768 MB RAM 768 MB RAM 
267 60 MB Dual Pentium {Dual Pentium | Ultra 60, Ultra 60, 
II 750, II 750, 1024 MB RAM | 1024 MB RAM 
1024 MB RAM | 1024 MB RAM 
445 100 MB Dual Pentium {Quad Pentium | E250, E250, 
II 750, II 750, 2048 MB RAM _ | 2048 MB RAM 
1024 MB RAM _ | 2048 MB RAM 
Tips and Tricks 


This section provides a number of tips and techniques you can use to improve 
RealServer’s performance with other system components. 


Layer-4 Load Balancing Switches 


Layer-4 switches currently do not recognize the UDP conversation sent from 
the client, if there is a re-send request due to lost packets. This happens 
because UDP data coming from the server to the client is sent to the client's 
IP address, bypassing the Virtual IP (VIP). The client still believes the server's 
IP address is the VIP, so it returns its UDP traffic to the VIP. The Layer-4 
switch (acting on the VIP port) does not recognize the UDP data, and drops 
the packet. Therefore, UDP data from the server to client works, but UDP 
traffic from the client to the server does not. This affects the quality-of-service 
for all clients accessing content through a switch. 
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NFS, 


RealServer now supports the RTSP transport parameter of the "source 
address" (as discussed in RFC 2326) . This feature enables the source 
RealServer to specify to the client what IP address it should use in sending 
back its UDP data. 


To enable this workaround, the RTSPSourceAddress configuration variable 
must be added to the server's configuration file: 


<Var RTSPSourceAddress="*"/ > 


If "*" is specified, then the server will insert the IP Address of the network 
interface that the client address came in on. 


Equally acceptable is designating a specific IP address: 
<Var RTSPSourceAddress="170.137.192.0"/> 


Note that this edit will require that you restart your server. 


Caching, and Content Management 


The Network File System (NFS) is a distributed file system that enables UNIX- 
or Solaris-based computers to access files and directories located on remote 
computers and to treat those files as if they were local. Often, deployments 
will include computers that are specifically designated to serve only streaming 
media. 


If possible, avoid using NFS as either a client or a server. It adds overhead and 
latency to the network, resulting in poorer performance. If you choose to use 
NFS and its caching functionality to manage your files—especially with 
content that needs to be updated frequently—be careful not to hard mount 
your paths, as this will result in the files being overwritten, with no 
mechanism available for controlling which file versions are the ones intended 
for distribution. 


Reporting 


Be sure to set parameters for access and error log files; otherwise, they will 
accumulate data indefinitely and consume system resources needlessly. Limit 
the log files either by time period or by size. You can set these restrictions in 
the Logging section in System Administrator, at 
http://yourserver:youradminport#/admin/config_logging.html. 
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Bandwidth Management 


You can set restrictions on streaming media bandwidth consumption in the 
Connection Control section of RealSystem Administrator (at 
http://yourserver:youradminport#/admin/config_allow.nc.html), where you 
can: 


* Control the number of RealPlayers that can receive streamed media from 
RealServer simultaneously. 


- Limit the amount of bandwidth used by RealServer. 


You can also distribute the stream capacity established in a valid license file to 
other RealServers (subscriber RealServers) that you operate. To do this, see the 
License Group section in RealSystem Administrator, at 
http://yourserver:youradminport#/admin/config_license.html. 


DNS Round-Robin 


Round-robin is a clustering technique that makes load sharing possible 
among multiple RealServers. All of the main RealServer platforms—Linux, 
Windows NT, and Solaris—support DNS and BIND. (DNS stands for Domain 
Name System, a hierarchical system that translates host names, or domain 
names, into Internet addresses.) 


This technique enables you to duplicate address records for a specific 
RealServer host, with different IP addresses. The domain name server then 
rotates addresses for any name that has multiple address records, thus 
handing off the load to an available RealServer as client requests are received. 


RealPlayer 


When installing RealPlayer® on a local area network (LAN) that has an ISDN 
router to the outside world, always be sure to disable RealPlayer’s Auto-Update 
functionality within the StartCenter feature. Otherwise, this feature may 
trigger the dial-on-demand feature in the router, resulting in continual, 
unnecessary dial-ups that could cost money and consume precious 


bandwidth. 
» To disable Auto-Update: 
1. In RealPlayer, click View>Preferences. 
2. On the General tab, click the Settings button under StartCenter. 


3. Clear the Enable StartCenter check box. 
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4. When asked whether you are sure you want to disable StartCenter, click 
Yes, and then click OK. 


5. Click OK in the Preferences dialog box. 


Determining the RealServer Version 
There are two methods for finding out which version of RealServer you are 
running. The following procedures explain these methods. 
> To determine the version programmatically: 


At the command prompt, navigate to the Bin directory, and then type the 
following: 


rmserver -V 


The version number appears, using the following syntax: 8.x. x. Xxx. 
> To determine the version by using RealSystem Administrator: 


In RealSystem Administrator, click About under Help in the table of contents 
area on the left. 


A new browser window appears, with information about your RealServer. The 
version number can vary, depending on the operating system you are using. 


Diagnosing RealServer Load Problems 


Use RealServer’s error messages to pinpoint problem areas. 


High Capacity Mode 
The “High Capacity mode” error message is a definite flag for performance 
problems, as it means that your computer is currently using more than 75 
percent of your system’s capacity. 
The RealServer error log is the first place you should go to review sluggish 
RealServer performance, especially if you are testing file performance. The 
error log’s default location in Windows NT is as follows: 
C:\ Program Files\ Real\ RealServer\ Logs\ rmerror.log 
For UNIX-based systems, there is no automatic default path, and the log file 
name Is..... /rmerror.log. 


The following is a sample error message: 
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*** 20-Oct-00 18:21:30.539 logplin(1571): The server has entered High Capacity 
mode, this may result in lowered quality of service for some players. The current 
load is xxx Mbps total output to xxx players (0% PNA, 100% RTSP, 0% HTTP, 0% 
TCP/ Cloaked, 100% UDP, 0% MCast). 


RealServer Error Messages 


For a comprehensive list of all of the error messages you might encounter 
when using RealServer, plus instructions for resolving them, see the following 
publication: 


http://service.real.com/help/library/guides/g270/srvrmsgs/srvrmsgs.ht 


m 


Other Troubleshooting Tips 


The RealServer Administration Guide offers a wide range of troubleshooting tips 
in Chapter 21. You will find this guide at 
http://service.real.com/help/library/guides/server8/realsrvr.htm. 


Whenever RealServer-specific security concerns arise, RealNetworks moves 
quickly to address them. The most recent security alerts are posted at 


http://service.real.com/help/faq/security/index.html. 


New Features in RealSystem Server 


This section discusses several of the key new features in RealSystem Server. 


Distributed Licensing 


This feature allows a set of RealServers within an organization to use the same 
license file. Features can be configured individually on separate RealServers, 
while sharing a pool of connections. The master license publisher will allocate 
its stream capacity equally among the license subscribers. Only one license 
publisher is active at any given time; additional license publisher hosts can be 
set up to provide redundancy in the event of a power outage or other network 
disruption. 


To determine the stream capacity of a given RealServer, click About: Summary 
in the table of contents area in RealSystem Administrator, and then view the 
numbers to the right of Max Client Connections. 
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Improved Codecs 


When used with RealProducer®, RealVideo® provides VHS quality for cable 
and DSL modems (500 Kbps) and near-DVD quality (1 Mbps). 


Port Hinting 


For those clients that can receive RealServer content only by way of the 
method known as “HTTP cloaking,” you can now create URLs that list the 
ports to try so that the client does not waste time by trying to discern the 
correct port. 


Redundant Encoders 


You can now use multiple sources as input for RealServer. With multiple 
sources, should one stream become unavailable, RealServer will automatically 
switch all users to the next available stream, thus ensuring a high quality of 
service for the users. 


Splitting 
The splitting feature has been completely rewritten to provide greater 
reliability, with some changes in terminology. What used to be called a source 
server is now called a transmitter, and what was called a splitter is now called a 
receiver. Also, note that multicasting between sources and splitters is now 
possible. As with RealServer 7, though, you can use splitting to deliver only live 
broadcasts; currently it does not work for on-demand broadcasts. 


The RealSystem Server Readme file provides extensive information about the 
changes in splitting functionality from version 7 to version 8. This file is 
available at 


http://service.real.com/help/library/guides/server8/readme.html. 


RealServer 7 License Key 


RealServer 7 license keys do not work on RealSystem Server 8. For more 
information on upgrading to RealSystem Server 8, visit the RealNetworks 
Customer Service Web site at http://service.real.com/faq/contcs.html, or 
call 1 (206) 674-2651. When RealNetworks releases a new major product 
version, we send out a new license to you via email. However, you must own a 
valid Upgrade and Support Contract at the time of the software release for 
you to receive the upgrade license file. If you have a valid Upgrade and 
Support agreement and did not receive the email, please contact Customer 
Service and we will send you the new license. 





10 


RealNetworks White Paper RealSystem Server Best Practices 





Additional Resources 


This section lists the documentation and other support options available for 
RealServer and associated RealNetworks products. 


Documentation 


RealServer Administration Guide 


This guide explains how to run the standard RealServer on whatever operating 
system (platform) you are using. You will find this book at 
http://service.real.com/help/library/guides/server8/realsrvr.htm. 


RealServer Readme File 


The Readme file flags late-breaking changes and other news related to 
RealServer. It also outlines general setup recommendations. You can view the 
latest version of this file at 
http://service.real.com/help/library/guides/server8/readme.html. 


Developing for RealServer's Web-based Management Application, the RealSystem 
Administrator 


This white paper explains how to customize RealSystem Administrator for 
your own or third-party plug-ins. You will find this paper at 
http://service.real.com/help/library/whitepapers/wpaper.html. 


Firewall Support 


Setting up RealServer to stream files past firewalls is a commonly performed 
task. Documentation on how to do this is presented at 
http://service.real.com/firewall/. 


RealSystem Production Guide 


This guide covers the nuts and bolts of creating streaming media and 
discusses authoring techniques you can use for optimum file performance. To 
view this book, go to 
http://service.real.com/help/library/guides/production8/realpgd.htm. 
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RealProducer Plus User's Guide 


This guide shows you how to use RealProducer to yield the highest possible 
quality of encoded content. You will find this book at 
http://service.real.com/help/library/encoders.html. 


Working with RealProducer 8 Codecs 


This document provides a detailed review of the significantly upgraded codecs 
that come with RealProducer 8. For a run-down of the best techniques for 
working with these codecs, see this blueprint at 


http://service.real.com/help/library/blueprints/8codecs/producer8code 
cs.html. 


RealProxy Administration Guide 


This guide fully documents RealProxy®. It explains how, in large-scale network 
deployments, RealServer works in tandem with RealProxy to reduce network 
traffic by eliminating redundant requests. You will find this book at 
http://service.real.com/help/library/guides/proxy/proxy.htm. 


Support Options 


Sometimes, despite your team’s best efforts, you will encounter a problem that 
you can not solve. RealNetworks offers two option for finding solutions to 
such problems: a comprehensive support program, and basic support. 


Platinum Support 


The Platinum Service Program for RealServer assigns a Technical Account 
Manager (TAM) to you. This person works closely with you to help solve your 
problems and address your issues. You can learn more about this program at 
http://www.realnetworks.com/products/servers/platinum.html. 


Regular Support 


You can call or e-mail our Technical Support team at any time. For the 
specifics, see http://service.real.com/help/call-html. 


RealForum 


Real Forum is a Web venue that hosts a moderated e-mail discussion group 
focused on using RealNetworks products. There you can post messages about 
the best methods you have found for creating content using RealNetworks 
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technologies or for planning and implementing streaming media. You can 
sign up at http://proforma.real.com/rn/misc/realforum/index.html. 


Appendix: Encoder-to-RealServer Parameters 


A crucial component of any broadcast network is the encoder. Your encoding 
platform captures, compresses, and formats your streaming media files. The 
encoded content can be sent directly to RealServer for streaming, or it can be 
saved for later broadcast. You can download RealProducer, RealNetworks’ 
streaming media authoring tool, from 
http://www.realnetworks.com/products/producerplus/info.html. 


The following tables provide a comprehensive overview of audio and video 
benchmarks for encoder-to-RealServer capacity parameters. (Note that 
RealNetworks SureStream™ technology is used in all of the scenarios 
presented in these tables.) These benchmarks illustrate the other end of the 
streaming equation—the beginning of the process of encoding your files and 
streaming them to RealServer. This information is especially useful if your 
encoder resides at a different, remote location and you need to establish a 
communication link back to the RealServer. 


These benchmarks are based on RealServer 7.02 numbers, but you can 
extrapolate them to apply also to RealProducer versions 8 and 8.5, as nothing 
has changed technically in the way media is streamed between encoders and 
RealServer. 


5:00 + Video: Normal Motion Using Standard SureStream—Voice Only 




















Codec Used Encoder Average 
reported measured Measured/ 
Single Dual rate bandwidth encoder 
28.8 KB 56KB ISDN ISDN DSL LAN |(Kbps) — (Kbps) (percent) 
x 37.00 52.00 141 
xX 69.90 100.00 143 
x 88.80 127.00 143 
xX 162.90 229.00 141 
384.30 466.00 121 
261.80 347.00 133 
x xX 66.00 87.00 132 
x xX x 111.00 141.00 127 























(Table Page 1 of 2) 
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5:00 + Video: Normal Motion Using Standard SureStream—Voice Only (continued) 








Codec Used Encoder Average 
reported measured Measured/ 
Single Dual rate bandwidth encoder 
28.8 KB 56KB ISDN ISDN DSL _ LAN | (Kbps) (Kbps) (percent) 
Xx Xx xX Xx 182.50 226.00 124 
Xx xX Xx xX 535.50 | 619.00 116 
Xx xX x xX 520.50 | 608.00 117 























(Table Page 2 of 2) 


5:00 + Video: Normal Motion Using Standard SureStream—Voice with Background 


















































Music 
Codec Used Encoder Average 
reported measured Measured/ 
Single Dual rate bandwidth encoder 
28.8 KB 56KB ISDN ISDN DSL _ LAN | (Kbps) (Kbps) (percent) 
Xx 34.00 47.00 138 
Xx 66.90 96.00 143 
xX 88.80 126.00 142 
x 162.90 229.00 141 
384.30 465.00 121 
261.80 344.00 131 
XxX x 61.50 80.00 130 
Xx x Xx 106.50 | 135.00 127 
Xx x x x 178.00 | 219.00 123 
xX Xx XxX x 531.00 611.00 115 
xX x x x 516.00 604.00 117 
3:00 + Audio: White Noise Using Standard SureStream—Voice Only 
Codec Used Encoder Average 
reported measured Measured/ 
Single Dual rate bandwidth encoder 
28.8 KB 56KB ISDN ISDN DSL _ LAN | (Kbps) (Kbps) (percent) 








(Table Page 1 of 2) 
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3:00 + Audio: White Noise Using Standard SureStream—Voice Only (continued) 























Codec Used Encoder Average 
reported measured Measured/ 
Single Dual rate bandwidth encoder 
28.8 KB 56KB ISDN ISDN DSL _ LAN | (Kbps) (Kbps) (percent) 
Xx 56.50 69.00 12.50 
x 112.10 129.00 16.90 
112.10 127.00 14.90 
112.10 128.00 15.90 
Xx x 31.00 40.00 9.00 
xX Xx Xx 63.00 77.00 14.00 
x xX Xx x 127.10 149.00 21.90 
Xx xX Xx xX 127.10 149.00 21.90 
xX Xx Xx Xx 127.10 149.00 21.90 























(Table Page 2 of 2) 


3:00 + Audio: White Noise Using Standard SureStream—Voice with Background 


























Music 
Codec Used Encoder Average 
reported measured Measured/ 
Single Dual rate bandwidth encoder 
28.8KB 56KB ISDN ISDN DSL LAN |(Kbps) — (Kbps) (percent) 
XxX 31.00 40.00 129 
Xx 56.50 68.00 120 
Xx 56.50 69.00 122 
x 112.10 129.00 115 
112.10 128.00 114 
112.10 127.00 113 
xX xX 63.00 77.00 122 
xX x Xx 63.00 77.00 122 
x xX x x 127.10 149.00 117 
x Xx Xx x 127.10 149.00 117 
xX xX x x 127.10 149.00 117 
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